1. Filed of the Invention
The present invention relates to an image processing method and an image processing apparatus which are used in progressive transfer of hierarchically stored image data and for carrying out processing, based on data at an intermediate hierarchical level being transferred, on hierarchical data to be transferred after the data at the intermediate hierarchical level have been transferred and/or on image data having been transferred, and also to a recording medium readable by a computer and storing a program to cause a computer to execute the processing.
2. Description of the Related Art
In the case where image data stored in an image server or the like are transferred to a terminal to be displayed on a monitor such as a CRT display, the image data are conventionally transferred through raster scan thereof. Therefore, an image is raster-displayed on the monitor. In other words, the image is sequentially reproduced on the monitor from the top to the bottom thereof. Likewise, when an image recorded in a recording medium such as a ZIP disc or an MO disc is displayed on a monitor, the image data are read from the recording medium and transferred to the monitor while being raster-scanned. In other words, the image is displayed while being raster-scanned. For this reason, in the field of medicine for example, when a patient""s radiation image or the like is transferred from an image server and displayed on a monitor, the content of the image is not understood until the entire image has been displayed even in the case where the image being transferred is unnecessary or wrong. As a result, confirmation of the content of an image is time-consuming and efficient diagnosis is not carried out.
Meanwhile, a photograph finishing system in a laboratory displays each image stored in a recording medium on a monitor and after that appropriateness of the image and necessity of correction are judged. Therefore, if image data are read from a recording medium and displayed on a monitor while being raster-scanned, confirmation of the content of the image after the entire image has been displayed is time-consuming and efficient printing cannot be carried out.
As a format for storing image data, various formats such as JPEG, GIF, and TIFF are used. Recently, another format has been proposed. According to this format, image data are stored in a hierarchy based on resolution or density resolution by using wavelet transform or the like and data at each level of the hierarchy (hierarchical data) are stored after coding and compression thereof. This storage format is called JPEG 2000 format. More specifically, image data are decomposed into hierarchical data according to multiple resolutions or density resolutions through wavelet transform or the like, and the hierarchical data at each resolution or density resolution are coded according to the hierarchy and saved as a file after compression thereof.
The JPEG 2000 method has the following characteristics:
(1) Unlike DCT (Discrete Cosine Transform) method used in the conventional JPEG format, no artifacts such as block distortion are created, since image data are not processed in each block.
(2) Only resolution information which is necessary for transfer of image data needs to be transferred, since the image data are to coded hierarchically. This enables efficient image transfer.
(3) Various kinds of image processing such as frequency enhancing processing can be carried out comparatively easily, since image data are decomposed into data at multiple resolutions or multiple density resolutions.
(4) Simultaneous decomposition of space and frequency according to multiple-resolution analysis is possible. In a low frequency range wherein coding is greatly affected, an orthogonal transform can be carried out in a wide range, whereas in a high frequency range, an orthogonal transform is possible in a narrow range. Therefore, even when quantization noise near an edge of an image is generated, spatial spread of the noise can be suppressed. As a result, the noise is not easy to perceive.
Like a FlashPix file proposed by Eastman Kodak Co., a file format enabling storage of data sets having different characteristics in a file has also been proposed. Hierarchical data decomposed into multiple resolutions or density resolutions can be stored in such a file according to the FlashPix standard.
When image data storing hierarchical images, such as in a JPEG 2000 file or a FlashPix file, are displayed on a monitor or the like, sequential data transfer to the monitor and image reproduction on the monitor from low resolution or low density resolution (hereinafter, simply expressed as resolution) to high resolution is carried out. This is called progressive transfer (or progressive reproduction in the case of reproduction). When the data transferred progressively are displayed on a monitor, an entire low resolution image is displayed first, and images with gradually improving sharpness are reproduced in accordance with increasing resolution of the data.
In the case of a FlashPix file, sequential display of transferred hierarchical image data leads to progressive reproduction of the images. In the case of a JPEG 2000 file, the transferred hierarchical data are first restructured into data enabling reproduction of an image at the resolution or density resolution according to the hierarchical level of the image and images are progressively reproduced based on the restructured data. As for the JPEG 2000 file, data enabling reproduction of an image can be generated from hierarchical data by a server storing the file and transferred from the server in some cases. The images are progressively reproduced by sequential reproduction of the transferred data from low resolution to high resolution.
An object of the present invention is therefore to provide an image processing method and an image processing apparatus for carrying out processing on image data transferred progressively in the manner as has been described above, and a computer-readable recording medium storing a program to cause a computer to execute the processing.
An image processing method of the present invention is an image processing method of carrying out processing, upon progressive transfer of hierarchically stored image data at each hierarchical level, on hierarchical data to be transferred progressively and/or on image data having been transferred. The image processing method of the present invention comprises the steps of:
determining the content of processing carried out, based on hierarchical data at a predetermined intermediate hierarchical level out of the hierarchical data at each hierarchical level transferred progressively, on hierarchical data to be transferred after the data at the intermediate hierarchical level and/or on the image data having been transferred, and
carrying out processing on the hierarchical data to be transferred and/or on the image data having been transferred, based on the determined content of the processing.
The term xe2x80x9cprogressive transferxe2x80x9d means transferring hierarchically stored image data structured according to multiple resolutions or density resolutions, sequentially from low resolution data to high resolution data. The data at each hierarchical level may have been coded or not coded. The hierarchical data to be transferred are different depending on a file format of the image data. In other words, in the case of a JPEG 2000 file (hereinafter called case 1), the data to be transferred progressively are data representing an edge component (high frequency component) at each resolution hierarchically structured by wavelet transform, or data representing an edge component in an image at each density resolution. In the case of a FlashPix file (hereinafter called case 2), the data to be transferred are data corresponding to a resolution or density resolution. In the case of a JPEG 2000 file storing hierarchical data at each hierarchical level to be restructured into data enabling reproduction of an image before being transferred (hereinafter called case 3), the data to be transferred progressively mean data corresponding to the resolution or density resolution, as in the case of the FlashPix file.
The xe2x80x9cintermediate level dataxe2x80x9d means hierarchical data transferred earlier than hierarchical data at a highest resolution or highest density resolution to be transferred last. The intermediate level data enable reproduction of an image at a resolution or density resolution lower than the highest resolution or density resolution. In other words, for the case 1 above, the intermediate level data means restructured data enabling reproduction of the transferred hierarchical data as an image. For the cases 2 and 3, the intermediate level data mean the hierarchical data having been transferred.
The xe2x80x9chierarchical data to be transferredxe2x80x9d mean hierarchical data representing an image at a resolution or density resolution higher than that of the intermediate level data.
The xe2x80x9cimage data having been transferredxe2x80x9d mean image data enabling reproduction of an original image. For the case 1 above, the image data having been transferred represent an image at the highest resolution or density resolution restructured from hierarchical data at all resolutions or density resolutions having been transferred progressively. For the cases 2 and 3, the image data having been transferred mean data at a highest resolution or density resolution having been transferred progressively.
It is preferable for the content of the processing to be processing to interrupt the progressive transfer of the hierarchical data to be transferred. After the progressive transfer of the hierarchical data to be transferred has been interrupted, the processing preferably resumes the transfer.
Furthermore, it is preferable for the content of the processing to be a condition of image processing carried out on the image data having been transferred.
An image processing apparatus of the present invention carries out processing, upon progressive transfer of hierarchically stored image data at each level, on hierarchical data to be transferred progressively and/or on image data having been transferred. The image processing apparatus comprises processing means for determining, based on hierarchical data at a predetermined intermediate hierarchical level out of the hierarchical data transferred progressively, the content of the processing on hierarchical data to be transferred after the predetermined intermediate level data and/or on the image data having been transferred, and for carrying out the processing on the hierarchical data to be transferred and/or on the image data having been transferred, based on the determined content of the processing.
More specifically, it is preferable for the processing means to comprise:
reproduction means for progressively reproducing the hierarchical data at each level transferred progressively;
interruption input means for receiving an input for interrupting the progressive transfer; and
interruption means for interrupting the progressive transfer when the interruption input is received by the interruption input means.
xe2x80x9cProgressively reproducingxe2x80x9d herein referred to means, for the case 1 above, restructuring images from the hierarchical data having been transferred progressively and displaying the images gradually from a low resolution or density resolution. For the cases 2 and 3, progressive reproduction means sequential display of transferred hierarchical data gradually from a low resolution or density resolution in the order of the transfer.
In this manner, the transferred hierarchical data at each level are reproduced progressively by the reproduction means and when the interruption input is received from the interruption input means, the progressive reproduction is suspended.
In this case, it is preferable for the processing means to comprise:
resumption input means for receiving an input to resume the progressive transfer; and
resumption means for resuming the progressive transfer in the case where the resumption input has been received by the resumption input means. In this manner, progressive transfer is resumed after the resumption input has been received by the resumption input means, and hierarchical data to be transferred after the predetermined intermediate level data are transferred.
It is preferable for the processing means to further comprise canceling means for canceling the progressive transfer when no input is received by the resumption input means within a predetermined amount of time after the interruption of the progressive transfer by the interruption means.
Furthermore, in this case, it is preferable for the image processing apparatus to further comprise image processing means for obtaining processed hierarchical data by carrying out predetermined image processing on the hierarchical data at each level reproduced progressively, based on resolution information thereof, and the reproduction means progressively reproduces the processed hierarchical data.
For the case 1, the hierarchical data subjected to image processing are obtained by restructuring the transferred hierarchical data as an image. For the cases 2 and 3, the hierarchical data subjected to the image processing mean the hierarchical data having been transferred.
In this case, the processed image data subjected to the predetermined image processing do not represent a high frequency component in accordance with the resolution of the data, but data enabling reproduction of an image at a resolution or density resolution lower than the highest resolution or density resolution.
In this manner, the hierarchical data at each level transferred progressively are reproduced progressively after image processing has been carried out thereon in accordance with the resolution thereof.
As the xe2x80x9cpredetermined image processingxe2x80x9d, unsharp masking processing described in Japanese Unexamined Patent Publication Nos. 55(1980)-163472, 55(1980)-87953, and 3(1991)-222577 and frequency processing such as dynamic range compression processing can be listed.
As the xe2x80x9cresolution informationxe2x80x9d, information on resolution of the hierarchical data (such as dpi and cycles/mm) is listed. If a file has a hierarchy of xc2xdn resolutions such as a wavelet transform and in the FlashPix format, information on the highest resolution and hierarchical levels can be listed as such information.
It is preferable for the processing means to comprise:
reproduction means for progressively reproducing the hierarchical data at each level transferred progressively; and
interruption means for interrupting the progressive transfer immediately after the predetermined intermediate level data have been transferred.
In this manner, the transferred hierarchical data at each level are reproduced progressively by the reproduction means and the progressive transfer is interrupted by the interruption means immediately after the predetermined intermediate level data have been transferred.
In this case, it is preferable for the processing means to comprise resumption input means for receiving an input for resumption of the progressive transfer; and
resumption means for resuming the progressive transfer when the resumption input is received by the resumption input means. In this manner, the progressive transfer is resumed when the resumption input is received by the resumption input means and hierarchical data to be transferred after the predetermined intermediate level data are transferred.
Moreover, it is preferable for the processing means to comprise canceling means for canceling the progressive transfer when the resumption input means receives no input within a predetermined amount of time after the interruption of the progressive transfer by the interruption means.
In this case, it is preferable for the image processing apparatus to further comprise:
image processing means for obtaining processed image data by carrying out predetermined image processing on the hierarchical data at each level reproduced progressively, based on resolution information of the progressively reproduced hierarchical data; and
the reproduction means is means for progressively reproducing the processed hierarchical data.
It is also preferable for the processing means to comprise:
image processing condition determining means for determining a condition of image processing carried out on the image data having been transferred, based on the predetermined intermediate level data; and
image processing means for carrying out image processing on the image data having been transferred, based on the image processing condition.
In this manner, the condition of the image processing based on the intermediate level data on the image data having been transferred is determined in parallel to the progressive transfer, after the predetermined intermediate level data have been transferred. When the image data having been transferred are obtained after the hierarchical data at all levels have been transferred, the processing according to the image processing condition is carried out on the image data having been transferred.
It is preferable for the processing means to comprise:
reproduction means for progressively reproducing the hierarchical data at each level transferred progressively;
interruption means for interrupting the progressive transfer immediately after the predetermined intermediate level data have been transferred;
image processing condition input means for receiving an input of a condition of image processing determined based on the predetermined intermediate level data and carried out on the image data having been transferred;
resumption input means for receiving an input for resumption of the progressive transfer;
resumption means for resuming the progressive transfer in the case where the resumption input means has received the resumption input; and
image processing means for carrying out image processing on the image data having been transferred, based on the image processing condition.
According to this processing means, the transferred hierarchical data at each level can be reproduced progressively by the reproduction means, and the progressive transfer is suspended by the interruption means after the predetermined intermediate level data have been transferred. An operator views the predetermined intermediate level data displayed by the reproduction means and inputs, by using the image processing condition input means, the image processing condition for the image data having been transferred, if necessary. Thereafter, when the resumption input is received by the resumption input means, the progressive transfer is resumed. When the image data having been transferred are received, image processing is carried out thereon based on the input image processing condition.
In this case, the resumption means preferably resumes the progressive transfer when no resumption input is received by the resumption input means within a predetermined amount of time after the interruption of the progressive transfer by the interruption means.
Furthermore, in this case, it is preferable for the image processing apparatus to comprise:
image processing means for obtaining processed image data by carrying out predetermined image processing on the hierarchical data at each level reproduced progressively, based on resolution information of the progressively reproduced hierarchical data, and
the reproduction means reproduces the processed image data progressively.
In this manner, the hierarchical data at each level can be reproduced progressively after image processing in accordance with the resolution thereof has been carried out thereon.
The processing carried out by each means above may be provided as a program recorded in a computer-readable recording medium to cause a computer to execute the processing.
According to the present invention, the content of the processing on the hierarchical data to be transferred after the predetermined intermediate level data and/or on the image data having been transferred is determined based on the intermediate level data out of the hierarchical data to be transferred progressively, and the processing based on the processing content is carried out on the hierarchical data to be transferred and/or on the image data having been transferred. For example, if the processing is interruption of the progressive transfer, transfer of the data to be transferred is interrupted, and an image obtained by reproduction of the intermediate level data corresponds to the resolution or density resolution of the intermediate level data. Therefore, an operator can judge the content of the image by viewing the image corresponding to the resolution or the density resolution of the intermediate level data, and confirmation of a transferred image can be carried out efficiently when the image transferred progressively is reproduced progressively.
If the transfer can be resumed after interruption thereof, when an operator judges an image to be necessary by viewing the image obtained by reproducing the intermediate level data, hierarchical data to be transferred can be transferred thereafter. Therefore, the transfer is canceled if the image data are not necessary, whereas the transfer is resumed if the data are necessary. In this manner, efficient transfer can be carried out.
In this case, the processed hierarchical data are obtained by carrying out image processing on the hierarchical data at each level transferred progressively, according to the resolution thereof. By progressively reproducing the processed hierarchical data, an image having the same frequency characteristic as an image obtained by image processing on the image data having been transferred can be obtained. Therefore, the image can be confirmed in the same manner as for an image at the highest resolution or density resolution. Consequently, judgment as to interruption or resumption of transfer can be carried out easily and efficient progressive transfer can be realized.
Furthermore, by determining the image processing to be carried out on the hierarchical data to be transferred based on the intermediate level data, progressive transfer can be carried out at the same time as determination of the processing condition. Therefore, image processing on the image data immediately after completion of the progressive transfer becomes possible. In this manner, efficient image processing can be carried out on the image data.